Accelerating the High-fidelity Simulation of Turbulence: Ensemble Averaging

Computational fluid dynamics (CFD) is increasingly used to simulate complex industrial systems. Most CFD analysis relies on the Reynolds-averaged Navier-Stokes (RANS) approach and traditional two-equation turbulence models. Higher-fidelity approaches to the simulation of turbulence such as wallresolved large eddy simulation (LES) and direct numerical simulation (DNS) remain limited to smaller applications or to large supercomputing platforms. Nonetheless, continued advances in supercomputing are enabling the simulation of physical systems of increasing size and complexity. These simulations can be used to gain unprecedented insight into the physics of turbulence in complex flows and will become more widespread as petascale architectures become more accessible. As the scale and size of LES and DNS simulations increase, however, the limitations of current algorithms become apparent. For larger systems, more temporal and spatial scale must be resolved, thus increasing the time-scale separation. While the smaller time scales dictate the size and the computational cost associated with each time step, the larger time scales dictate the length of the transient. An increased time-scale separation leads to smaller time steps and longer transients, eventually leading to simulations that are impractical or infeasible. In practice the presence of multiple and strongly separated time scales limits the effectiveness of CFD algorithms for LES and DNS applied to large industrial systems. Moreover, the situation is likely to become worse on future computer architectures, as even larger systems will be simulated, thus increasing the size and length of transients. At the same time transients currently simulated on petascale architectures are unlikely to become any faster on exascale architectures. In this paper we consider a technique to accelerate current transient simulations aimed at collect averaged turbulent statistics. The focus is on ergodic flows and simulations. This technique is ensemble averaging, commonplace in machine learning and artificial neural networks. Ensemble averaging is the process of creating multiple models and combining them to produce a desired output. It is also at the basis of RANS/URANS turbulence modeling. In the proposed approach, multiple instances of the same ergodic flows are simulated in parallel for a short timeframe and summed to create an ensemble. Provided each instance is sufficiently statistically decorrelated, this allows considerable reduction in the time to solution. This paper focuses on the theory and implementation of the methodology in Nek5000, a massively parallel open-source spectral element code. Moreover, we present the application of the method to the DNS and LES simulation of channel flow and pipe flow. INTRODUCTION Computational fluid dynamics (CFD) is increasingly used to simulate turbulent flows. Most CFD analysis, especially in industry, relies on the Reynolds-averaged Navier-Stokes (RANS) approach and traditional two-equation turbulence models. Higher-fidelity approaches to the simulation of turbulence, such as wall-resolved large eddy simulation (LES) and direct numerical simulation (DNS), remain limited to smaller applications or to large supercomputing platforms. In fact, since the Reynolds number dictates the local resolution, large machines are currently necessary to simulate engineering systems with turbulence-resolving techniques. Nonetheless, continued advances in supercomputing are enabling the simulation of physical systems of increasing size and complexity. Current supercomputers can accommodate grids that reach tens to hundreds of billions of points, enabling the simulation of entire rod bundles with wall-resolved LES using CFD algorithms with good scalability properties. As supercomputer become more powerful and larger, LES and DNS simulations become possible. With traditional algorithms, however, little can be done to make the solution of traditional problems run faster once the strong-scaling limit is reached. Therefore, the time to solution of traditional algorithms is